Baby Mental Life: Study 1

“Baby Mental Life: Study 1” was conducted on MTurk on 2018-07-31 through 2018-08-01.

Our planned sample was 300 participants, and we anticipated that roughly 85-90% of recruited participants would pass all of our attention checks, so we initially recruited 342 participants (on the idea that ~87.5% of 342 ~ 300 participants; note that for administrative purposes we need to recuit participants in batches that were divisible by 9). After filtering out participants who failed at least one of our attention checks, we ended up retaining only 80% of participants, so we recruited an additional 36 participants for a total of 378 people recruited (on the idea that ~80% of 378 ~ 300 participants; note that for administrative purposes we need to recuit participants in batches that were divisible by 9). In a slight deviation from our preregistered recruitment plan, we limited these additional 36 participants to women, because we had an unexpectedly large imbalance of men to women.

UPDATE: On 2018-08-09, KW discovered that there were many repeating GPS locations, a symptom of a recent outbreak of bots on MTurk. As of 2019-05-15, these data INCLUDE participants where there is another participant with an identical set of GPS coordinates as recorded by Qualtrics. Excluding these participants would exclude 76 participants (25%).

In the end, we ended up with a sample of 301 participants who passed our attention checks, 225 of whom came from unique GPS coordinates.

Each participant assessed children’s mental capacities at 3 target ages: newborns, 9-month-olds, and 5-year-olds. For each target, they rated 60 mental capacities on a scale from 0 (not at all capable) to 100 (completely capable).

For more details about the study, see our preregistration here.

── Attaching packages ─────────────────────────────────────── tidyverse 1.2.1 ──
✔ ggplot2 3.1.1     ✔ purrr   0.2.5
✔ tibble  2.1.1     ✔ dplyr   0.7.6
✔ tidyr   0.8.1     ✔ stringr 1.3.1
✔ readr   1.1.1     ✔ forcats 0.3.0
package ‘ggplot2’ was built under R version 3.5.2package ‘tibble’ was built under R version 3.5.2── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()

Attaching package: ‘langcog’

The following object is masked from ‘package:base’:

    scale


Attaching package: ‘psych’

The following objects are masked from ‘package:ggplot2’:

    %+%, alpha

Loading required package: Matrix

Attaching package: ‘Matrix’

The following object is masked from ‘package:tidyr’:

    expand

Data preparation

NAs introduced by coercionattributes are not identical across measure variables;
they will be droppedJoining, by = "question_qualtrics"

NAs introduced by coercion

EFA: all targets

Our primary analysis is an exploratory factor analysis (EFA) collapsing across all 3 target characters (and treating an individual participant’s responses to each character as if they were independent data points) - see the preregistration for more details.

We planned to examine three factor retention protocols in order to determine how many factors to retain: Parallel analysis, minimizing BIC, and a set of preset criteria outlined in Weisman et al. (2017). Here we look at each solution in turn.

Rotation choices

We planned to examine oblimin-rotated solutions (which allow factors to correlate), but you could examine other rotation options by selecting a different rotation type here.

Parallel analysis

How many factors to retain?

Parallel analysis suggests that the number of factors =  4  and the number of components =  3 
Call: fa.parallel(x = d_all, plot = F)
Parallel analysis suggests that the number of factors =  4  and the number of components =  3 

 Eigen Values of 

What are these factors?

Loading required namespace: GPArotation
 A loading greater than abs(1) was detected.  Examine the loadings carefully.

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to which targets?

Ignoring unknown aesthetics: y

Here’s a multilevel linear regression on these factor scores, with random intercepts and slopes (for target and factor) by participant. Target is coded with linear and quadratic contrasts, and factor is effect-coded for comparison with the grand mean (with the last factor as the base, -1):

Linear mixed model fit by REML ['lmerMod']
Formula: score ~ target * factor + (target + factor | ResponseId)
   Data: efa_all_par_scores

REML criterion at convergence: 7036.7

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-3.7531 -0.5165  0.0146  0.5656  3.1771 

Random effects:
 Groups     Name        Variance Std.Dev. Corr                         
 ResponseId (Intercept) 0.29003  0.5385                                
            target.L    0.16960  0.4118   -0.26                        
            target.Q    0.01948  0.1396   -0.84  0.37                  
            factor1     0.20636  0.4543   -0.44 -0.52  0.19            
            factor2     0.29750  0.5454    0.26  0.68  0.11 -0.83      
            factor3     0.03002  0.1733   -0.17 -0.14  0.26  0.30  0.14
 Residual               0.22433  0.4736                                
Number of obs: 3612, groups:  ResponseId, 301

Fixed effects:
                   Estimate Std. Error t value
(Intercept)      -1.279e-15  3.203e-02   0.000
target.L          7.567e-01  2.738e-02  27.636
target.Q          5.780e-02  1.584e-02   3.648
factor1           1.350e-15  2.953e-02   0.000
factor2          -1.712e-15  3.427e-02   0.000
factor3          -3.760e-17  1.691e-02   0.000
target.L:factor1  3.775e-01  2.364e-02  15.966
target.Q:factor1  2.295e-01  2.364e-02   9.706
target.L:factor2 -3.939e-01  2.364e-02 -16.663
target.Q:factor2 -9.555e-02  2.364e-02  -4.041
target.L:factor3  2.949e-01  2.364e-02  12.474
target.Q:factor3 -2.535e-01  2.364e-02 -10.724

If we consider t-values > 2 to be “significant” and use the plot above to guide our interpretation, here’s what we conclude:

• [(Intercept): By definition, because we’re using factor scores, which are similar to z-scores, the intercept is ~0.]
• target.L: Overall, mental capacity attributions increase with target age.
• target.Q: This increase is generally non-linear.
• [factor1, factor2, …: By definition, because we’re using factor scores, which are similar to z-scores, the differences in means across factors are ~0.]
• target.L:factor1: The linear increase is more dramatic (steeper slope) for MR1, compared to other factors.
• target.Q:factor1: The nonlinearity is more dramatic (curvier and/or concave instead of convex curve) for MR1, compared to other factors.
• target.L:factor2: The linear increase is less dramatic (shallower slope) for MR2, compared to other factors.
• target.Q:factor2: The nonlinearity is less dramatic (flatter curve) for MR2, compared to other factors.
• target.L:factor3: The linear increase is more dramatic (steeper slope) for MR3, compared to other factors.
• target.Q:factor3: The nonlinearity is less dramatic (flatter curve) for MR3, compared to other factors.

Joining, by = "capacity"

|=====================================                 | 69% ~1 s remaining     
|=====================================                 | 70% ~1 s remaining     
|======================================                | 71% ~1 s remaining     
|=======================================               | 72% ~1 s remaining     
|=======================================               | 74% ~1 s remaining     
|========================================              | 75% ~1 s remaining     
|=========================================             | 76% ~1 s remaining     
|=========================================             | 77% ~1 s remaining     
|==========================================            | 78% ~1 s remaining     
|==========================================            | 79% ~1 s remaining     
|===========================================           | 81% ~1 s remaining     
|============================================          | 82% ~1 s remaining     
|=============================================         | 84% ~1 s remaining     
|==============================================        | 86% ~0 s remaining     
|===============================================       | 87% ~0 s remaining     
|===============================================       | 88% ~0 s remaining     
|================================================      | 90% ~0 s remaining     
|=================================================     | 91% ~0 s remaining     
|=================================================     | 92% ~0 s remaining     
|==================================================    | 93% ~0 s remaining     
|==================================================    | 94% ~0 s remaining     
|===================================================   | 96% ~0 s remaining     
|====================================================  | 97% ~0 s remaining     
|====================================================  | 98% ~0 s remaining     
|===================================================== | 98% ~0 s remaining     
|===================================================== | 99% ~0 s remaining     
|===================================================== | 99% ~0 s remaining     

Joining, by = c("capacity", "factor", "order")

Minimizing BIC

How many factors to retain?

Very Simple Structure
Call: vss(x = x, n = n, rotate = rotate, diagonal = diagonal, fm = fm, 
    n.obs = n.obs, plot = plot, title = title, use = use, cor = cor)
VSS complexity 1 achieves a maximimum of 0.94  with  1  factors
VSS complexity 2 achieves a maximimum of 0.99  with  2  factors

The Velicer MAP achieves a minimum of 0.01  with  6  factors 
BIC achieves a minimum of  -6013.88  with  6  factors
Sample Size adjusted BIC achieves a minimum of  -1658.73  with  8  factors

Statistics by number of factors 

What are these factors?

convergence not obtained in GPFoblq. 1000 iterations used. A loading greater than abs(1) was detected.  Examine the loadings carefully.

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to which targets?

Ignoring unknown aesthetics: y

Here’s a multilevel linear regression on these factor scores, with random intercepts and slopes (for target and factor) by participant. Target is coded with linear and quadratic contrasts, and factor is effect-coded for comparison with the grand mean (with the last factor as the base, -1):

If we consider t-values > 2 to be “significant” and use the plot above to guide our interpretation, here’s what we conclude:

• [(Intercept): By definition, because we’re using factor scores, which are similar to z-scores, the intercept is ~0.]
• target.L: Overall, mental capacity attributions increase with target age.
• target.Q: This increase is not generally non-linear.
• [factor1, factor2, …: By definition, because we’re using factor scores, which are similar to z-scores, the differences in means across factors are ~0.]
• target.L:factor1: The linear increase is more dramatic (steeper slope) for MR1, compared to other factors.
• target.Q:factor1: The nonlinearity is more dramatic (curvier and/or concave instead of convex curve) for MR1, compared to other factors.
• target.L:factor2: The linear increase is less dramatic (shallower slope) for MR2, compared to other factors.
• target.Q:factor2: The nonlinearity for MR2 is comparable to other factors.
• target.L:factor3: The linear increase is more dramatic (steeper slope) for MR3, compared to other factors.
• target.Q:factor3: The nonlinearity is less dramatic (flatter curve) for MR3, compared to other factors.
• target.L:factor4: The linear increase is less dramatic (shallower slope) for MR4, compared to other factors.
• target.Q:factor4: The nonlinearity is more dramatic (curvier and/or concanve instead of convex curve) for MR4, compared to other factors.
• target.L:factor5: The linear increase is more dramatic (steeper slope) for MR5, compared to other factors.
• target.Q:factor5: The nonlinearity is less dramatic (flatter curve) for MR5, compared to other factors.

Joining, by = "capacity"

|===============================================       | 88% ~0 s remaining     
|================================================      | 91% ~0 s remaining     
|==================================================    | 93% ~0 s remaining     
|===================================================   | 95% ~0 s remaining     
|====================================================  | 98% ~0 s remaining     

Joining, by = c("capacity", "factor", "order")

How do these compare to parallel analysis?

Joining, by = "capacity"

Preset retention criteria

How many factors to retain?

[1] "Preset criteria suggest retaining 2 factors"

What are these factors?

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to which targets?

Ignoring unknown aesthetics: y

Here’s a multilevel linear regression on these factor scores, with random intercepts and slopes (for target and factor) by participant. Target is coded with linear and quadratic contrasts, and factor is effect-coded for comparison with the grand mean (with the last factor as the base, -1):

unable to evaluate scaled gradientModel failed to converge: degenerate  Hessian with 1 negative eigenvalues

Linear mixed model fit by REML ['lmerMod']
Formula: score ~ target * factor + (target + factor | ResponseId)
   Data: efa_all_k_scores

REML criterion at convergence: 3395.6

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-3.6194 -0.5160 -0.0066  0.5597  2.6303 

Random effects:
 Groups     Name        Variance Std.Dev. Corr             
 ResponseId (Intercept) 0.2968   0.5448                    
            target.L    0.2112   0.4595   -0.31            
            target.Q    0.0134   0.1158   -1.00  0.25      
            factor1     0.1565   0.3957   -0.37 -0.64  0.41
 Residual               0.1652   0.4065                    
Number of obs: 1806, groups:  ResponseId, 301

Fixed effects:
                   Estimate Std. Error t value
(Intercept)      -1.688e-16  3.283e-02   0.000
target.L          9.083e-01  3.124e-02  29.074
target.Q          3.140e-02  1.786e-02   1.758
factor1           1.392e-15  2.473e-02   0.000
target.L:factor1  2.596e-01  1.657e-02  15.671
target.Q:factor1  1.729e-01  1.657e-02  10.437
convergence code: 0
unable to evaluate scaled gradient
Model failed to converge: degenerate  Hessian with 1 negative eigenvalues

There are some problems with this model that I haven’t addressed - see warnings above. Ignoring them for now…

If we consider t-values > 2 to be “significant” and use the plot above to guide our interpretation, here’s what we conclude:

• [(Intercept): By definition, because we’re using factor scores, which are similar to z-scores, the intercept is ~0.]
• target.L: Overall, mental capacity attributions increase with target age.
• target.Q: This increase is generally non-linear.
• [factor1: By definition, because we’re using factor scores, which are similar to z-scores, the differences in means across factors are ~0.]
• target.L:factor1: The linear increase is more dramatic (steeper slope) for MR1, compared to other factors.
• target.Q:factor1: The nonlinearity is more dramatic (curvier and/or concave instead of convex curve) for MR1, compared to other factors.

Joining, by = "capacity"

|================================================      | 89% ~0 s remaining     
|=================================================     | 92% ~0 s remaining     
|==================================================    | 94% ~0 s remaining     
|===================================================   | 96% ~0 s remaining     
|===================================================== | 98% ~0 s remaining     

Joining, by = c("capacity", "factor", "order")

How do these compare to parallel analysis?

Joining, by = "capacity"

EFA: newborns

What happens if we limit ourselves to assessments of newborns’ mental capacities?

Parallel analysis

How many factors to retain?

Parallel analysis suggests that the number of factors =  4  and the number of components =  3 
Call: fa.parallel(x = d_00mo, plot = F)
Parallel analysis suggests that the number of factors =  4  and the number of components =  3 

 Eigen Values of 

What are these factors?

 A loading greater than abs(1) was detected.  Examine the loadings carefully.

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to newborns?

Joining, by = "capacity"
Joining, by = c("capacity", "factor", "order")

Minimizing BIC

How many factors to retain?

Very Simple Structure
Call: vss(x = x, n = n, rotate = rotate, diagonal = diagonal, fm = fm, 
    n.obs = n.obs, plot = plot, title = title, use = use, cor = cor)
VSS complexity 1 achieves a maximimum of 0.86  with  1  factors
VSS complexity 2 achieves a maximimum of 0.98  with  2  factors

The Velicer MAP achieves a minimum of 0.01  with  4  factors 
BIC achieves a minimum of  -6124.8  with  4  factors
Sample Size adjusted BIC achieves a minimum of  -1385.1  with  8  factors

Statistics by number of factors 

What are these factors?

 A loading greater than abs(1) was detected.  Examine the loadings carefully.

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to newborns?

Joining, by = "capacity"
Joining, by = c("capacity", "factor", "order")

Preset retention criteria

How many factors to retain?

[1] "Preset criteria suggest retaining 2 factors"

What are these factors?

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to newborns?

Joining, by = "capacity"
Joining, by = c("capacity", "factor", "order")

EFA: 9-month-olds

What happens if we limit ourselves to assessments of 9-month-olds’ mental capacities?

Parallel analysis

How many factors to retain?

Parallel analysis suggests that the number of factors =  4  and the number of components =  2 
Call: fa.parallel(x = d_09mo, plot = F)
Parallel analysis suggests that the number of factors =  4  and the number of components =  2 

 Eigen Values of 

What are these factors?

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to 9-month-olds?

Joining, by = "capacity"
Joining, by = c("capacity", "factor", "order")

Minimizing BIC

How many factors to retain?

Very Simple Structure
Call: vss(x = x, n = n, rotate = rotate, diagonal = diagonal, fm = fm, 
    n.obs = n.obs, plot = plot, title = title, use = use, cor = cor)
VSS complexity 1 achieves a maximimum of 0.86  with  1  factors
VSS complexity 2 achieves a maximimum of 0.98  with  2  factors

The Velicer MAP achieves a minimum of 0.01  with  6  factors 
BIC achieves a minimum of  -5996.91  with  4  factors
Sample Size adjusted BIC achieves a minimum of  -1366.77  with  8  factors

Statistics by number of factors 

What are these factors?

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to 9-month-olds?

Joining, by = "capacity"
Joining, by = c("capacity", "factor", "order")

Preset retention criteria

How many factors to retain?

[1] "Preset criteria suggest retaining 2 factors"

What are these factors?

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to 9-month-olds?

Joining, by = "capacity"
Joining, by = c("capacity", "factor", "order")

EFA: 5-year-olds

What happens if we limit ourselves to assessments of 5-year-olds’ mental capacities?

Parallel analysis

How many factors to retain?

Parallel analysis suggests that the number of factors =  3  and the number of components =  2 
Call: fa.parallel(x = d_60mo, plot = F)
Parallel analysis suggests that the number of factors =  3  and the number of components =  2 

 Eigen Values of 

What are these factors?

 A loading greater than abs(1) was detected.  Examine the loadings carefully.

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to 5-year-olds?

Joining, by = "capacity"
Joining, by = c("capacity", "factor", "order")

Minimizing BIC

How many factors to retain?

Very Simple Structure
Call: vss(x = x, n = n, rotate = rotate, diagonal = diagonal, fm = fm, 
    n.obs = n.obs, plot = plot, title = title, use = use, cor = cor)
VSS complexity 1 achieves a maximimum of 0.96  with  1  factors
VSS complexity 2 achieves a maximimum of 0.99  with  2  factors

The Velicer MAP achieves a minimum of 0.01  with  6  factors 
BIC achieves a minimum of  -4946.45  with  5  factors
Sample Size adjusted BIC achieves a minimum of  -470.82  with  8  factors

Statistics by number of factors 

What are these factors?

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to 5-year-olds?

Joining, by = "capacity"
Joining, by = c("capacity", "factor", "order")

Preset retention criteria

How many factors to retain?

[1] "Preset criteria suggest retaining 2 factors"

What are these factors?

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Which capacities are attributed to 5-year-olds?

Joining, by = "capacity"
Joining, by = c("capacity", "factor", "order")

Subset of capacities from Weisman et al. (2017)

Parallel analysis suggests that the number of factors =  3  and the number of components =  2 

Very Simple Structure
Call: vss(x = x, n = n, rotate = rotate, diagonal = diagonal, fm = fm, 
    n.obs = n.obs, plot = plot, title = title, use = use, cor = cor)
VSS complexity 1 achieves a maximimum of 0.91  with  1  factors
VSS complexity 2 achieves a maximimum of 0.99  with  2  factors

The Velicer MAP achieves a minimum of 0.01  with  3  factors 
BIC achieves a minimum of  -995.35  with  5  factors
Sample Size adjusted BIC achieves a minimum of  -249.51  with  8  factors

Statistics by number of factors 

[1] 2

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

Demographics

Joining, by = "ResponseId"

Joining, by = "ResponseId"

Joining, by = "ResponseId"

Joining, by = "ResponseId"

Joining, by = "ResponseId"

Save EFA output

Finally, I’ll export the 4-factor EFA results to a saved file that I can draw on in our Study 2 analyses.